Method and system for detecting changes in road-layout information

ABSTRACT

A method for detecting changes in road information includes converting, by a domain adapter, a captured road image and a projected road-layout map into first intermediate data and second intermediate data of a same feature space, respectively, calculating, by a similarity determiner, a similarity between the captured road image and the projected road-layout map based on the first intermediate data and the second intermediate data, and detecting, by a change detector, presence or absence of changes in road information on the projected road-layout map based on the calculated similarity.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority under 35 U.S.C § 119 to Korean PatentApplication No. 10-2020-0051754, filed in the Korean IntellectualProperty Office on Apr. 28, 2020, the entire contents of which arehereby incorporated by reference.

BACKGROUND OF THE INVENTION Field of Invention

The present disclosure relates to a method and system for detectingchanges in road information, and more specifically, to a method andsystem for sensing or detecting whether or not there are changes in roadinformation included in a high-definition map using a high-definitionmap and a captured road image.

Description of Related Art

In recent years, as interest and importance of autonomous drivingvehicles or road driving information increases, construction ofhigh-definition (HD) maps or road-layout maps including informationnecessary for the vehicle to travel, such as road lanes, road markers,and the like is in progress. In general, the road-layout maps areproduced using high-definition aerial photographs, and the road-layoutmaps are generated and stored in the form of vector data by extractinginformation on necessary road or structure from the aerial photographs.

Since the accuracy of the road-layout map is determined by whether ornot the current road status or information is reflected, the road-layoutmap needs to be kept up-to-date by rapidly detecting and correctingchanges in road lanes, road markers, and the like represented on themap. However, periodically acquiring high-definition aerial photographsto detect changes that may occur at any time on the road-layout map isinefficient when considering high cost. In addition, even when theaerial photographs are acquired, it is difficult to detect a change bydirectly comparing the aerial photograph stored in the form of an imagewith the road-layout map previously stored in the form of vector data.

Meanwhile, the related algorithms for detecting changes in images ordata require that the data to be compared be expressed as data pairsarranged in the same form. In addition, among these change detectionalgorithms, algorithms developed to detect the changes in roadinformation of high-definition maps stored in an electronic format alsorequire a combination of expensive and various sensors to collect datato be compared, and can be applied only to predefined road information.Therefore, there is a problem in that it is difficult to detect a changeby directly inputting a previously stored road-layout map and a newlyacquired aerial photograph into a change detection algorithm withoutpre-processing.

BRIEF SUMMARY OF THE INVENTION

The present disclosure is designed to solve the problems describedabove, and provides a method and system for sensing and detectingchanges in road information on a road-layout map through a road imagecaptured using a low-cost camera rather than an expensive aerialphotograph.

The present disclosure provides a method and system for sensing anddetecting changes in road information without a pre-processing processof aligning or converting the data in analyzing and comparing aroad-layout map stored in the form of vector data with a captured roadimage in another data format.

In addition, the present disclosure provides a method and system forcomparing information of a previously stored road-layout map with acurrently captured road image to detect and display a region where achange occurs on the corresponding map.

The present disclosure may be implemented in a variety of ways,including a method, a system, a device, or a computer program stored ina non-transitory computer-readable recording medium.

The method for detecting changes in road information performed by atleast one processor of a computer system according to an embodiment isprovided. The method includes: converting, by a domain adapter, acaptured road image and a projected road-layout map into firstintermediate data and second intermediate data of a same feature space,respectively, calculating, by a similarity determiner, a similaritybetween the captured road image and the projected road-layout map basedon the first intermediate data and the second intermediate data, anddetecting, by a change detector, presence or absence of changes in roadinformation on the projected road-layout map based on the calculatedsimilarity.

A method for training an artificial neural network for detecting changesin road information performed by at least one processor of a computersystem according to another embodiment is provided. The method includestraining an adversarial learning artificial neural network to performdomain adaptation between a captured road image and a projectedroad-layout map by converting the captured road image and the projectedroad-layout map into first intermediate data and second intermediatedata of a same feature space, respectively, training a metric learningartificial neural network to extract features of the captured road imageand the projected road-layout map based on the first intermediate dataand the second intermediate data, and calculate a distance between theextracted features as a similarity between the captured road image andthe projected road-layout map, and training a change detectionartificial neural network to detect presence or absence of changes inroad information on the projected road-layout map based on thecalculated similarity.

According to another embodiment, there is provided a non-transitorycomputer-readable recording medium storing instructions for executing,on a computer, the method for detecting changes in road information orthe method for training an artificial neural network for detectingchanges in road information described above.

A system for detecting changes in road information according to anotherembodiment is provided. The system includes a domain adapter configuredto convert a captured road image and a projected road-layout map intofirst intermediate data and second intermediate data of a same featurespace, respectively, to perform domain adaptation between the capturedroad image or the projected road-layout map, a similarity determinerconfigured to calculate a similarity between the captured road image andthe projected road-layout map based on the first intermediate data andthe second intermediate data, and a change detector configured to detectpresence or absence of changes in road information on the projectedroad-layout map based on the calculated similarity.

According to some embodiments, it is possible to detect the presence orabsence of changes in road information on the road-layout map based onthe image captured by the camera that is available at a relatively lowcost compared to a high-definition aerial photograph.

According to some embodiments, by detecting a difference between theprojected road-layout map and the image captured by a camera, it ispossible to automatically detect a region where a change occurs in roadinformation on the road-layout map. By correcting the road informationon the road-layout map according to the detected changes in roadinformation as described above, it is possible to maintain theup-to-dateness of the road-layout map at low cost.

According to some embodiments, in order to sense or detect changes inroad information on the road-layout map, it is not necessary to performa separate pre-processing process such as aligning data to be comparedwith the road-layout map or converting the form of data. Accordingly, bysimply inputting a road image captured by a camera and the road-layoutmap of a corresponding location, the presence or absence of changes inroad information and a region of such changes in the road informationcan be easily sensed or detected.

According to some embodiments, it is possible to artificially generatethe data needed to train an artificial neural network for detectingchanges in road information on the road-layout map. Therefore, when itis difficult to acquire a dataset with actual changes in roadinformation, such as a high-definition map, it is possible to solve aproblem of difficulty of training the artificial neural network todetect the changes in road information. Further, since it is possible totrain the artificial neural network using data including variousartificially generated changes in road information, the performance ofartificial neural networks can be further enhanced in detecting changesin road information on the road-layout map.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present disclosure will be described with referenceto the accompanying drawings described below, where similar referencenumerals indicate similar elements, but are not limited thereto, inwhich:

FIG. 1 is a schematic diagram illustrating a configuration in which adigital map database, a camera, and a system for detecting changes inroad information are communicatively connected to each other in order toprovide a service for detecting changes in road information according toan embodiment;

FIG. 2 is an exemplary diagram illustrating an operation indicating aninput of a system for detecting changes in road information and aresultant output according to an embodiment;

FIG. 3 is a block diagram illustrating an internal configuration of acomputing device and an information processing system according to anembodiment;

FIG. 4 is a block diagram illustrating an internal configuration of asystem for detecting changes in road information according to anembodiment;

FIG. 5 is a block diagram illustrating an internal configuration of asystem for detecting changes in road information according to anotherembodiment;

FIG. 6 is an exemplary diagram illustrating an input of a map projectorof a system for detecting changes in road information and a resultantoutput according to an embodiment;

FIG. 7 is a block diagram illustrating an internal configuration of anencoder and a feature extractor according to an embodiment;

FIG. 8 is an exemplary diagram illustrating an operation of generatingtraining data for a system for detecting changes in road informationaccording to an embodiment;

FIG. 9 is a block diagram illustrating an internal configuration of adomain adapter according to an embodiment;

FIG. 10 is a block diagram illustrating an internal configuration of achange region detector according to an embodiment;

FIG. 11 is an exemplary diagram illustrating a change value mapgenerated based on a captured road image and a projected road-layout mapby a system for detecting changes in road information according to anembodiment;

FIG. 12 is an exemplary diagram illustrating a change value mapgenerated based on a captured road image and a projected road-layout mapby a system for detecting changes in road information according toanother embodiment;

FIG. 13 is a graph illustrating a result of performance of detectingchanges in road information of a system for detecting changes in roadinformation according to an embodiment;

FIG. 14 is a flowchart illustrating a method for detecting changes inroad information according to an embodiment; and

FIG. 15 is a flowchart illustrating a method for training an artificialneural network for detecting changes in road information according to anembodiment.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, specific details for the practice of the present disclosurewill be described in detail with reference to the accompanying drawings.However, in the following description, detailed descriptions ofwell-known functions or configurations will be omitted when it may makethe subject matter of the present disclosure rather unclear.

In the accompanying drawings, the same or corresponding elements areassigned the same reference numerals. In addition, in the followingdescription of the embodiments, duplicate descriptions of the same orcorresponding components may be omitted. However, even if descriptionsof elements are omitted, it is not intended that such elements are notincluded in any embodiment.

Advantages and features of the disclosed embodiments and methods ofaccomplishing the same will be apparent by referring to embodimentsdescribed below in connection with the accompanying drawings. However,the present disclosure is not limited to the embodiments disclosedbelow, and may be implemented in various different forms, and thepresent embodiments are merely provided to make the present disclosurecomplete, and to fully disclose the scope of the invention to thoseskilled in the art to which the present disclosure pertains.

The terms used herein will be briefly described prior to describing thedisclosed embodiments in detail. The terms used herein have beenselected as general terms which are widely used at present inconsideration of the functions of the present disclosure, and this maybe altered according to the intent of an operator skilled in the art,conventional practice, or introduction of new technology. In addition,in a specific case, a term is arbitrarily selected by the applicant, andthe meaning of the term will be described in detail in a correspondingdescription of the embodiments. Therefore, the terms used in the presentdisclosure should be defined based on the meaning of the terms and theoverall contents of the present disclosure rather than a simple name ofeach of the terms.

As used herein, the singular forms “a,” “an,” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesthe singular forms. Further, the plural forms are intended to includethe singular forms as well, unless the context clearly indicates theplural forms.

Further, throughout the description, when a portion is stated as“comprising (including)” a component, it intends to mean that theportion may additionally comprise (or include or have) anothercomponent, rather than excluding the same, unless specified to thecontrary.

Further, the term “module” or “unit” used herein refers to a software orhardware component, and “module” or “unit’ performs certain roles.However, the meaning of the “module” or “unit” is not limited tosoftware or hardware. The “module” or “unit” may be configured to be inan addressable storage medium or configured to execute at least oneprocessor. Accordingly, as an example, the “module” or “unit” mayinclude components such as software components, object-oriented softwarecomponents, class components, and task components, and at least one ofprocesses, functions, attributes, procedures, subroutines, program codesegments of program code, drivers, firmware, micro-codes, circuits,data, database, data structures, tables, arrays, and variables.Furthermore, functions provided in the components and the “modules” or“units” may be combined into a smaller number of components and“modules” or “units”, or further divided into additional components and“modules” or “units.”

In the present disclosure, “road information” may include variousinformation necessary for a vehicle to travel on a road. For example,the road information may include road lanes including a white dottedline, a while solid line, a yellow line, a blue line, a stop line, andthe like, arrow markers including a straight line, a left turn, a rightturn, a U-turn, and a prohibition, texts including a no waiting zone, acrosswalk, a speed bump, a speed limit, a yield, and the like, and/orinformation on various structures including image information,guardrails, overpasses, buildings, and the like arranged on or aroundthe road.

In the present disclosure, a “domain” may refer to target data to betrained by a machine learning model such as an artificial neuralnetwork, or a distribution or range of such data. For example, when theartificial neural network is trained to detect changes in the roadinformation based on image or image information, an image of a road orits surroundings captured by a camera may form a domain having a uniquecharacteristic or distribution. In addition, when the artificial neuralnetwork is trained to detect changes in the road information based onroad object information expressed in form of vector data, ahigh-precision map or road-layout map expressed in the form of vectordata, or a two-dimensional or three-dimensional image generated based onthe map may form another domain.

In the present disclosure, “domain adaptation” may refer to a techniqueof transferring from a domain of data that the machine learning modelsuch as the artificial neural network is trained to classify orrecognize to another domain or adapting to another domain. For example,when the machine learning model is trained to recognize, or detect thechange of, the road information based on the domain of the road imagecaptured by the camera, the domain adaptation or transfer learning maybe performed to perform the same or similar inference function for theroad-layout map expressed in the form of vector data or the domain of animage generated based on the map.

FIG. 1 is a schematic diagram illustrating a configuration 100 in whicha digital map database 120, a camera 140, and a system 180 for detectingchanges in road information are communicatively connected to each otherin order to provide a service for detecting changes in road informationaccording to an embodiment.

As illustrated in FIG. 1, the system 180 for detecting changes in roadinformation may be connected to the digital map database (DB) 120 andthe camera 140 through a network 160. The system 180 for detectingchanges in road information may process a road-layout map received froma digital map DB 120 and a captured road image received from the camera140 to sense or extract changes in road information on the road-layoutmap. According to an embodiment, the system 180 for detecting changes inroad information may receive a road-layout map corresponding to acorresponding location from the digital map DB 120 based on locationinformation of a road image captured by the camera 140. In addition, thesystem 180 for detecting changes in road information may project aroad-layout map into an image using a location and camera parametersprovided by the camera 140. The system 180 for detecting changes in roadinformation may detect changes in road information on the road-layoutmap based on the captured road image received from the camera 140 andthe projected road-layout map. The system 180 for detecting changes inroad information may output a value representing changes in the roadinformation on the road-layout map, or may output data or an imagerepresenting a corresponding region of changes in the road information.

The system 180 for detecting changes in road information may include oneor more computing devices and/or databases capable of storing,providing, and executing computer executable programs (e.g., adownloadable application) and data for providing a road informationchange detection function or one or more distributed computing devicesand/or distributed databases based on cloud computing services.

The network 160 may be configured to enable communication between thesystem 180 for detecting changes in road information and the digital mapDB 120 and the camera 140. The network 160 may be configured as a wirednetwork 160 such as Ethernet, a wired home network (Power LineCommunication), a telephone line communication device and RS-serialcommunication, a wireless network 160 such as a mobile communicationnetwork, a wireless LAN (WLAN), controller area network (CAN), Wi-Fi,Bluetooth, and ZigBee, or a combination thereof, depending on theinstallation environment. The communication method is not limited, andmay include not only a communication method using a communicationnetwork (e.g., mobile communication network, wired Internet, wirelessInternet, broadcasting network, satellite network, and the like) thatmay be included in the network 160, but also short-range wirelesscommunication between the camera and an external device (e.g., aposition sensor associated with a camera, a memory or a car with acamera, and the like) associated with the camera. For example, thenetwork 160 may include any one or more of networks including a personalarea network (PAN), a local area network (LAN), a campus area network(CAN), a metropolitan area network (MAN), a wide area network (WAN), abroadband network (BBN), the Internet, and the like. In addition, thenetwork 160 may include any one or more of network topologies includinga bus network, a star network, a ring network, a mesh network, astar-bus network, a tree or hierarchical network, and the like, but notlimited thereto.

The digital map DB 120 may store high-definition map data includinginformation necessary for a vehicle to travel, such as road lanes, roadmarkers, and the like. For example, the digital map DB 120 may be storedin the form of vector data including location information generatedbased on high-definition aerial photographs, spatial coordinates of roadlanes, road markers, and the like, and geometric information (connectioninformation such as point, line, plane, polygon, and the like) of thecomponents. In the digital map DB 120, each of road lanes, road markers,and the like may be stored as one object, and each of the objects may beclassified by a class or an identifier.

Map data stored in the digital map DB 120 in the form of vectors may beexpressed in the form of a road-layout map in two-dimensional orthree-dimensional representation. In the present embodiment, a“road-layout map” may refer to at least a part of a digital map or ahigh-definition map including map information stored in the form ofvector. More specifically, the road-layout map may refer to a map inwhich information about each lane unit such as a road center line and aroad boundary line, and the like, and information about crosswalks,signs, curbs, road markers, various structures, and the like, are storedin the form of vector. For example, the road-layout map may includespatial coordinates (x, y, z), type, and connectivity information forroad lanes and road markers. In the road-layout map, each of theinterconnected road lanes and road markers can be expressed as a singleobject, and for example, the road marker may be represented by a closedpolygon, and the road lane may be represented by a straight linecrossing the center. The digital map DB 120 may transmit the road-layoutmap corresponding to the location information of the captured road image144 to the system 180 for detecting changes in road information.

In FIG. 1, the digital map DB 120 is located outside the system 180 fordetecting changes in road information and communicates with the system180 for detecting changes in road information through the network 160,but is not limited thereto. For example, the digital map DB 120 may beincluded in the system 180 for detecting changes in road information.

The camera 140 for capturing the road image 144 may include a sensor ordevice capable of capturing an image of, or an image that can identifythe road information such as road lanes, road markers, and the like,such as a digital camera, a vehicle black box camera, a closed circuittelevision (CCTV), and the like. The location information of thecaptured road image 144 may be acquired through an image-basedpositioning algorithm, a location sensor, or the like. To this end, thecamera 140 may include a sensor such as a GPS receiver which is capableof recognizing the location information of the captured road image 144.Alternatively, the camera 140 may be associated with a sensor that ispresent outside the camera 140 and capable of recognizing the locationinformation of the captured road image 144. For example, a sensor thatis present outside of the camera 140 and capable of recognizing thelocation information of the captured road image 144 may be included in amobile device to which the camera 140 is attached. In this example, thelocation information may include a location coordinate (x, y, z) fromwhich an image is acquired and a rotation angle (θ) of the camera in thex-y plane. In addition, the camera 140 may provide the unique cameraparameters which were applied when generating the captured road image.The camera parameters may include an eigenvalue (K), a rotation matrix(R), a transition matrix (T), and the like.

The system 180 for detecting changes in road information may receive,from the camera, 140 the captured road image 144, the locationinformation of the captured road image 144, and/or the captured roadimage information including the camera parameters. For example, thesystem 180 for detecting changes in road information may be directlyconnected to the camera 140 or connected via a network 160 to receivethe captured road image information. In another example, the capturedroad image information provided by the camera 140 may be stored in anexternal memory connected to the camera 140, and the system 180 fordetecting changes in road information may receive the captured roadimage information from the external memory.

As illustrated in FIG. 1, the camera 140 for capturing a road image maybe attached to a vehicle 142 or the like and carried around, but is notlimited thereto, and may include one or more cameras fixed at apredetermined position. Further, in FIG. 1, the system 180 for detectingchanges in road information is illustrated to be connected to one camera140, but is not limited thereto, and the system 180 for detectingchanges in road information may be connected to a plurality of cameras140.

FIG. 2 is an exemplary diagram illustrating an operation 200 indicatingan input of the system 180 for detecting changes in road information anda resultant output according to an embodiment.

According to the operation 200 illustrated in FIG. 2, when a capturedroad image 220 received from the camera 140 and a road-layout map 240received from the digital map DB 120 are input to the system 180 fordetecting changes in road information, the presence or absence 260 ofchanges in the road information may be determined by identifying asimilarity or difference between the captured road image 220 and theroad-layout map 240 and may be output. Here, the road-layout map 240 maybe extracted from the digital map DB 120 based on the locationinformation of the captured road image 220. For example, the system 180for detecting changes in road information may receive the road-layoutmap 240 corresponding to the region within k meters around the location(where k is any preset number) from the digital map DB 120 by using thelocation information of the captured road image 220 provided by thecamera 140.

The system 180 for detecting changes in road information may determinethe presence or absence 260 of changes in the current road informationof a corresponding location in the map data (e.g., road lanes, roadmarkers, and the like as the road information) previously stored in thedigital map DB 120 by comparing the captured road image 220 and theroad-layout map 240. For example, the system 180 for detecting changesin road information may determine what changes (e.g., addition,deletion, change of an object, and the like) occur in the current roadinformation of the corresponding location in the map data previouslystored in the digital map DB 120. As another example, the system 180 fordetecting changes in road information may determine a region where theroad information change occurs on the road-layout map (e.g., a location,a size of the region, and the like at which the road information changeoccurs).

FIG. 3 is a block diagram illustrating an internal configuration of acomputing device 310 and an information processing system 340 accordingto an embodiment. The computing device 310 may refer to any devicecapable of capturing and/or processing an image of a road or itssurroundings and capable of wired/wireless communication, and mayinclude the camera 140, the vehicle 142, and the like of FIG. 1, forexample. As illustrated, the computing device 310 may include a memory312, a processor 314, a communication module 316, and an input andoutput interface 318. Likewise, the information processing system 340(e.g., the system 180 for detecting changes in road information) mayinclude a memory 342, a processor 344, a communication module 346, andan input and output interface 348. As shown in FIG. 3, the computingdevice 310 and the information processing system 340 may be configuredto communicate information and/or data through the network 330 using therespective communication modules 316 and 346. In addition, the input andoutput device 320 may be configured to input information and/or data tothe computing device 310 or to output information and/or data generatedfrom the computing device 310 through the input and output interface318.

The memories 312 and 342 may include any non-transitorycomputer-readable recording medium. According to an embodiment, thememories 312 and 342 may include a permanent mass storage device such asrandom access memory (RAM), read only memory (ROM), disk drive, solidstate drive (SSD), flash memory, and the like. As another example, anon-destructive mass storage device such as ROM, SSD, flash memory, diskdrive, and the like may be included in the computing device 310 or theinformation processing system 340 as a separate permanent storage devicethat is separate from the memory. In addition, an operating system andat least one program code may be stored in the memories 312 and 342.

These software components may be loaded from a computer-readablerecording medium separate from the memories 312 and 342. Such a separatecomputer-readable recording medium may include a recording mediumdirectly connectable to the computing device 310 and the informationprocessing system 340, and may include a computer-readable recordingmedium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, amemory card, and the like, for example. As another example, the softwarecomponents may be loaded into the memories 312 and 342 through thecommunication modules 316 and 346 rather than the computer-readablerecording medium. For example, at least one program may be loaded intothe memories 312 and 342 based on a computer program installed by filesprovided by developers or a file distribution system that distributes aninstallation file of an application through the network 330.

The processors 314 and 344 may be configured to process instructions ofthe computer program by performing basic arithmetic, logic, and inputand output operations. The instructions may be provided to theprocessors 314 and 344 from the memories 312 and 342 or thecommunication modules 316 and 346. For example, the processors 314 and344 may be configured to execute the received instructions according toprogram code stored in a recording device such as the memories 312 and342.

The communication modules 316 and 346 may provide a configuration orfunction for the computing device 310 and the information processingsystem 340 to communicate with each other through the network 330, andmay provide a configuration or function for the computing device 310and/or the information processing system 340 to communicate with anothercomputing device 310 or another system (e.g., a separate cloud system orthe like). For example, the requests or data generated by the processor314 of the computing device 310 according to the program code stored inthe recording device such as the memory 312 or the like may betransmitted to the information processing system 340 through the network330 under the control of the communication module 316. Conversely, acontrol signal or instructions provided under the control of theprocessor 344 of the information processing system 340 may be receivedby the computing device 310 through the communication module 316 of thecomputing device 310 via the communication module 346 and the network330.

The input and output interface 318 may be a means for interfacing withthe input and output device 320. As an example, the input device of theinput and output device 320 may include a device such as a camera, akeyboard, a microphone, and a mouse, which includes an image sensorand/or an audio sensor, and the output device of the input and outputdevice 320 may include a device such as a display, a speaker, a hapticfeedback device, and the like. As another example, the input and outputinterface 318 may be a means for interfacing with a device such as atouch screen or the like that integrates a configuration or function forperforming inputting and outputting. For example, when the processor 314of the computing device 310 processes the instructions of the computerprogram loaded in the memory 312, a service screen or the like, which isconfigured with the information and/or data provided by the informationprocessing system 340 or other computing devices may be displayed on thedisplay through the input and output interface 318. While FIG. 3illustrates that the input and output device 320 is not included in thecomputing device 310, embodiment is not limited thereto, and it may beconfigured as one device with the computing device 310. In addition, theinput and output interface 348 of the information processing system 340may be a means for interfacing with a device (not illustrated) forinputting or outputting, which may be connected to the informationprocessing system 340 or included in the information processing system340. In FIG. 3, the input and output interfaces 318 and 348 areillustrated as the components configured separately from the processors314 and 344, but are not limited thereto, and the input and outputinterfaces 318 and 348 may be configured to be included in theprocessors 314 and 344, respectively.

The computing device 310 and the information processing system 340 mayinclude more components than the components illustrated in FIG. 3.However, it would be unnecessary to exactly illustrate all of therelated components. According to an embodiment, the computing device 310may be implemented to include at least some of the input and outputdevices 320 described above. In addition, the computing device 310 mayfurther include other components such as a global positioning system(GPS) module, a Lidar, a wheel encoder, an inertial measurement unit(IMU), a transceiver, various sensors, a database, and the like.

According to an embodiment, the processor 314 of the computing device310 may be configured to extract image data or features of an image. Inthis case, an associated program code may be loaded into the memory 312of the computing device 310. While the program code is running, theprocessor 314 of the computing device 310 may receive information and/ordata provided from the input and output device 320 through the input andoutput interface 318 or receive information and/or data from theinformation processing system 340 through the communication module 316,and may process the received information and/or data and store it in thememory 312. In addition, such information and/or data may be provided tothe information processing system 340 through the communication module316.

The processor 314 may receive images or selected text, and the likethrough input devices such as an image sensor, a GPS module, a touchscreen, a keyboard, an audio sensor, and the like connected to theinput/output interface 318, and may store the received images, locationinformation, camera parameters, text, and the like in the memory 312 orprovide them to the information processing system 340 through thecommunication module 316 and the network 330. In an embodiment, theprocessor 314 may provide the captured image, the location information,and the camera parameters received through the input device to theinformation processing system 340 through the network 330 and thecommunication module 316. Alternatively, the processor 314 may extractthe features of the captured image and provide the same to theinformation processing system 340 together with the location informationand the camera parameters.

The processor 344 of the information processing system 340 may beconfigured to manage, process, and/or store the information and/or thedata received from a plurality of computing devices and/or a pluralityof external systems. According to an embodiment, the processor 344 maygenerate a projected road-layout map and/or information or a mapindicating the presence or absence of changes in the road information ora region of such changes based on the data received from the computingdevice 310. Additionally, or alternatively, the processor 344 maytransmit a projected road-layout map and/or information or a mapindicating the presence or absence of changes in road information or aregion of such changes, and the like to the computing device 310.

FIG. 4 is a block diagram illustrating an internal configuration of thesystem 180 for detecting changes in road information according to anembodiment. As illustrated, the system 180 for detecting changes in roadinformation may include a map projector 420, a domain adapter 440, asimilarity determiner 460, and a change detector 480. In addition, thedomain adapter 440 may include two encoders 442_1 and 442_2 and adiscriminator 444, and the similarity determiner 460 may include twofeature extractors 462_1 and 462_2 and a similarity calculator 464.

The map projector 420 may receive the location information and thecamera parameters of the captured road image from the camera 140, andmay receive the road-layout map from the digital map DB 120. The mapprojector 420 may project the road-layout map using the locationinformation and the camera parameters of the captured road image. Themap projector 420 may use a known algorithm for projectingthree-dimensional world coordinates onto a two-dimensional image planein projecting the road-layout map. Specifically, the map projector 420may extract all road-layout objects within k meters around the locationby using the location information (e.g., x, y, z coordinates) of thecaptured road image. For example, in order to solve the problem ofdifficulty of projection into image because the road lane object isrepresented by lines among the road-layout objects, the map projector420 may perform a dilation operation on the road lane object and convertit into polygon data having a predetermined width. In addition, in orderto prevent errors that may occur during the projection of theroad-layout map, the map projector 420 may remove objects within apredetermined distance in front of the camera 140 in the road-layout mapby using the rotation angle of the camera. The map projector 420 may beconfigured to input the projected road-layout map to the second encoder442_2, which is one of the two encoders 442_1 and 442_2 included in thedomain adapter 440.

The domain adapter 440 may be configured to convert the captured roadimage and the projected road-layout map into first intermediate data andsecond intermediate data of the same feature space, respectively. Thatis, the domain adapter 440 may be configured to perform the domainadaptation between the captured road image or the projected road-layoutmap. According to an embodiment, the first encoder 442_1 included in thedomain adapter 440 may convert the captured road image into the firstintermediate data, and the second encoder 442_2 included therein mayconvert the projected road-layout map into the second intermediate data.For example, the first intermediate data and the second intermediatedata may correspond to data of the same feature space in the form offour-dimensional tensor.

The discriminator 444 included in the domain adapter 440 may beconfigured to receive the first intermediate data and the secondintermediate data from each of the encoders 442_1 and 442_2, andcalculate a domain difference between the first intermediate data andthe second intermediate data. According to an embodiment, a differencebetween an expected value of an output value of the discriminator forfirst intermediate data and an expected value of an output value of thediscriminator for second intermediate data may be calculated as thedomain difference. When the first encoder 442_1 and the second encoder442_2 of the domain adapter 440 are implemented as an artificial neuralnetwork-based machine learning model which will be described below, thefirst encoder 442_1 and the second encoder 442_2 may be trained tominimize a difference between the domain of the first intermediate dataand the domain of the second intermediate data calculated by thediscriminator 444.

The similarity determiner 460 may include the feature extractors 462_1and 462_2 configured to extract the features of each of the capturedroad image and the projected road-layout map, and the similaritycalculator 464 that calculates similarity based on the extracted featurevalues. The feature extractors 462_1 and 462_2 may receive the firstintermediate data and the second intermediate data from the encoders442_1 and 442_2 of the domain adapter 440, respectively. The featureextractors 462_1 and 462_2 may extract the features of each of thecaptured road image and the road-layout map based on the received firstintermediate data and the second intermediate data. For example, thefirst intermediate data and the second intermediate data received fromthe encoders 442_1 and 442_2 of the domain adapter 440 may correspond todata in the form of four-dimensional tensor, and the features extractedfrom the feature extractors 462_1 and 462_2 may correspond to data inthe form of one-dimensional feature vector.

The similarity calculator 464 of the similarity determiner 460 maycalculate the similarity by using the feature value of the captured roadimage extracted through the feature extractors 462_1 and 462_2 and thefeature value of the projected road-layout map. For example, thesimilarity calculator 464 may calculate the similarity through the dotproduct of the feature values in the form of one-dimensional featurevector received from the feature extractors 462_1 and 462_2. As anotherexample, the similarity calculator 464 may calculate the similaritythrough correlation between the feature values in the form ofone-dimensional feature vector received from the feature extractors462_1 and 462_2.

The similarity determiner 460 may include a metric learning artificialneural network configured to calculate a distance between the featuresextracted from the captured road image and the road-layout map as asimilarity. To this end, a metric learning artificial neural network maybe configured to calculate a similarity between the captured road imageand the projected road-layout map using a triplet loss function thatenforces a similarity between the captured road image and a positiveimage which is the projected road-layout map corresponding to thecaptured road image to be larger by a margin than a similarity betweenthe captured road image and a negative image which is the projectedroad-layout map not corresponding to the captured road image. Meanwhile,the margin adapted for the triplet loss function of the metric learningartificial neural network may be adjusted according to the domaindifference between the first intermediate data and the secondintermediate data calculated by the domain adapter 440. For example, inthe initial stage of learning where the domain difference between thetwo input data of the metric learning artificial neural network, thecaptured road image and the projected road-layout map, is large, themargin is decreased, so that learning is still performed with littleloss. On the other hand, in the training phase in which the domaindifference between the captured road image and the projected road-layoutmap is decreased, the margin is increased and the training is performedso that it converges to the margin designed for the metric learningartificial neural network. As described above, by adjusting the marginof the triplet loss function according to the domain difference betweenthe two data inputs to the metric learning artificial neural network orthe degree of domain adaptation, the metric learning artificial neuralnetwork can be trained to identify or detect the difference with highaccuracy even for the input data having different domains.

In training a metric learning artificial neural network, when a specificobject-displaying region in either the positive image or the negativeimage is occluded by another moving object (e.g., a moving vehicle or apedestrian) in the captured road image, the image and the captured roadimage may not be used for training. That is, an image in which a regionwhere a specific object is displayed among the positive image or thenegative image is occluded by another dynamic object in the capturedroad image may not be used as training data of the metric learningartificial neural network. Through this training method, when the system180 for detecting changes in road information detects changes in theroad information, an error due to a dynamic object may be reduced.

The change detector 480 may be configured to detect the presence orabsence of changes in the road information on the projected road-layoutmap based on the similarity calculated by the similarity calculator 464of the similarity determiner 460. The change detector 480 may calculatea degree of change of the road information on the projected road-layoutmap through the similarity value calculated by the similarity calculator464. For example, when the similarity calculator 464 calculates thesimilarity by dot product (x_(l) ^(T)·x_(M)) of the feature values inthe form of a one-dimensional feature vector, a value (1−x_(l)^(T)·x_(M)) obtained by subtracting the similarity from 1 may be definedas the degree of change.

When the similarity value calculated by the similarity calculator 464 isequal to or greater than a preset threshold value, the change detector480 may determine that there is no change in the current roadinformation as compared with the projected road-layout map. On the otherhand, when the calculated similarity value is equal to or less than apreset threshold value, the change detector 480 may determine that thechange has occurred in the current road information as compared with theprojected road-layout map.

In FIG. 4, although the system 180 for detecting changes in roadinformation is illustrated as including the map projector 420, thedomain adapter 440, the similarity determiner 460, and the changedetector 480, it is not limited thereto, and may include more componentsthan the components of FIG. 4. In addition, although it is illustratedthat the similarity determiner 460 and the change detector 480 areseparate from each other, the present disclosure is not limited thereto,and the change detector 480 may be included in the similarity determiner460.

FIG. 5 is a block diagram illustrating an internal configuration of thesystem 180 for detecting changes in road information according toanother embodiment.

The system 180 for detecting changes in road information may include themap projector 420, the domain adapter 440, the similarity determiner460, and the change detector 480. Since the map projector 420, thedomain adapter 440, and the similarity determiner 460 have beendescribed in detail with reference to FIG. 4, redundant descriptionsthereof will be omitted below. The change detector 480 may include achange region detector 502.

During the process of extracting the features of the captured road imageand the projected road-layout map by the feature extractors 462_1 and462_2 of the similarity determiner 460, third intermediate data andfourth intermediate data may be generated, respectively. The thirdintermediate data and the fourth intermediate data thus generated may betransmitted to the change region detector 502 of the change detector480.

The change region detector 502 of the change detector 480 may receivethe third intermediate data and the fourth intermediate data from thefeature extractors 462_1 and 42_2 of the similarity determiner 460,respectively. The third intermediate data and the fourth intermediatedata received by the change region detector 502 may correspond to dataof a different form from the feature values received from the featureextractors 462_1 and 462_1 in order for the similarity calculator 464 tocalculate the similarity between the captured road image and theprojected road-layout map, as described with reference to FIG. 4. Forexample, the third intermediate data and the fourth intermediate datamay be at least some of intermediate data used by the feature extractors462_1 and 462_1 to output a final feature value.

The change region detector 502 may generate a change value map based onthe received third intermediate data and the fourth intermediate data.Here, the generated change value map may refer to a map displaying aregion of change on the projected road-layout map compared with thecaptured road image. In addition, the change value map may refer to animage displaying a region of change projected on the same space as theprojected road-layout map or the captured road image. For example, whenan object displayed on the projected road-layout map is not in acorresponding region in the captured road image, or when an object notdisplayed on the projected road-layout map is present in the capturedroad image, or when an object other than the object displayed on theprojected road-layout map is present in a corresponding region in thecaptured road image, the corresponding regions may be displayed on thechange value map. In addition, the region displayed on the change valuemap may be displayed in different colors according to the type of change(e.g., deletion, change, addition of object).

In FIG. 5, although the system 180 for detecting changes in roadinformation is illustrated as including the map projector 420, thedomain adapter 440, the similarity determiner 460, and the changedetector 480, it is not limited thereto, and may include more componentsthan the components of FIG. 5.

FIG. 6 is an exemplary diagram illustrating an input of the mapprojector 420 of the system 180 for detecting changes in roadinformation and a resultant output according to an embodiment.

As described above, the system 180 for detecting changes in roadinformation may include the map projector 420. The map projector 420 mayreceive the road-layout map extracted from the digital map DB 120according to the location coordinates (x, y, z) of the captured roadimage received by the system 180 for detecting changes in roadinformation. For example, the map projector 420 may extract and receivea road-layout map 620 corresponding to a range of surrounding k metersfrom a corresponding location in the road-layout map extracted from thedigital map DB 120. In addition, the map projector 420 may extract aroad-layout object included in the surrounding k meters from thecorresponding location from the digital map DB 120 according to thelocation coordinates (x, y, z) of the captured road image received bythe system 180 for detecting changes in road information. Here, thereceived road-layout map 620 may include coordinates, type, connectivityinformation, and the like for the location of a road lane, or a roadmarker, and each of the connected lanes, road marker, and the like maycorrespond to one object. In the road-layout map 620, the road markersmay be represented by closed polygons, and the road lane may berepresented by the straight line crossing the center.

The map projector 420 may cut out the objects corresponding to anydistance in front of the camera by using the camera rotation angle toprevent errors that may occur when projecting the road-layout map 620.Further, in consideration of the fact that the width of the actual laneis 20 cm, the lane object represented as a straight line in theroad-layout map 620 may be converted into polygon data having a width of20 cm through an expansion operation.

The map projector 420 may project the road-layout objects into the imagespace using the received location information and the camera parameter640. That is, as in the illustrated operation 600, the map projector 420may project the three-dimensional road-layout map 620 into atwo-dimensional image space that is same as that of the captured roadimage, to generate the projected road-layout map 660.

As illustrated in FIGS. 4 and 5, the map projector 420 may be includedin the system 180 for detecting changes in road information, but is notlimited thereto, and the map projector 420 may be present outside thesystem 180 for detecting changes in road information and connected tothe system for detecting changes in road information. For example, themap projector 420 may receive the road-layout map 620 from the digitalmap DB 120 outside of the system 180 for detecting changes in roadinformation and transmit the projected road-layout map 660 to the system180 for detecting changes in road information.

FIG. 7 is a block diagram illustrating an internal configuration of theencoder 442 and the feature extractor 462 according to an embodiment.

The system 180 for detecting changes in road information may include ametric learning artificial neural network configured to measure asimilarity between a captured road image and a projected road-layoutmap. Here, the system 180 for detecting changes in road information mayuse the artificial neural network model illustrated in FIG. 7 to extractthe feature values of each of the captured road image and the projectedroad-layout map in order to measure the similarity between the capturedroad image and the projected road-layout map. FIG. 7 illustrates onlythe configuration of the artificial neural network model correspondingto one encoder 442 and one feature extractor 462 for convenience ofexplanation, but this may include two encoders including theconfiguration of the same or similar artificial neural network modelsthat receive the captured road image and the projected road-layout maprespectively, and two feature extractors corresponding thereto.

According to the artificial neural network model structure illustratedin FIG. 7, the encoder 442 is an encoder of a recursively generativeadversarial neural network (CycleGAN), which may include a firststructure 710 including a reflection padding layer, a convolution layer,an instance normalization layer, and a rectified linear unit/leakedrectified linear unit (ReLU/LeakyReLU) layer in order, a secondstructure 720 including a convolution layer, an instance normalizationlayer, and a rectified linear unit/leaked rectified linear unit layer inorder, a third structure 730 including the same layers as the secondstructure, and a fourth structure 740 including a reflection paddinglayer, a convolution layer, an instance normalization layer, a rectifiedlinear unit/leaked rectified linear unit, a reflection padding layer,the convolution layer, and an instance normalization layer in order. Theencoder 442 may be configured to pass the input captured road image orprojected road-layout map through the first structure 710, the secondstructure 720, and the third structure 730, and repeatedly pass throughthe fourth structure 740 nine times. Each of the two encoders 442 mayoutput first intermediate data and second intermediate data in the formof four-dimensional tensor.

The two feature extractors 462 may receive the first intermediate dataor the second intermediate data in the form of four-dimensional tensorfrom the two encoders 442, respectively. The feature extractor 462 mayinclude a first structure 750 in the same form as the fourth structure740 of the encoder 442, a second structure 760 including a reflectionpadding layer, a convolution layer, an instance normalization layer, anda rectified linear unit/leaked rectified linear unit layer in order, anda generalized mean (GeM) pooling layer, and an instance normalizationlayer. Here, the first structure 750 may refer to a residual block. Thefeature extractor 462 may be configured to repeatedly pass the inputfirst intermediate data or second intermediate data through the firststructure 750 three times, add the data output from the first structure750 to the data passed through the second structure 720 once and thedata passed through the second structure 720 three times, and pass theresult through the generalized mean (GeM) pooling layer 770 and theobject normalization layer 780. Each of the two feature extractors 462may output a feature value (x) in the form of one-dimensional featurevector.

Here, the convolution layers included in the configuration of theencoder 442 and the feature extractor 462 may perform convolutionaccording to the values of different parameters (e.g., input size (i),output size (o), kernel (k), pooling size (p), stride (s)).

FIG. 8 is an exemplary diagram illustrating an operation of generatingtraining data for the system 180 for detecting changes in roadinformation according to an embodiment.

The metric learning artificial neural network included in the similaritydeterminer 460 is configured to calculate a similarity between acaptured road image and a projected road-layout map using the tripletloss function that enforces a similarity between the captured road imageand a positive image which is the projected road-layout mapcorresponding to the captured road image to be greater by a margin thana similarity between the captured road image and a negative image whichis the projected road-layout map not corresponding to the captured roadimage. For example, the loss function used for training the metriclearning artificial neural network can be expressed as the followingequation.

Loss(Loss Function)=max (0, (d(x _(r) , x _(p))+m−d(x _(r) , x _(n))))

where x_(r) is a feature value of the captured road image, x_(p) is afeature value of the positive image, x_(n) is a feature value of thenegative image, and d(x, y) is a distance between x and y. That is, asthe distance between the images or the features of the image decreases,it may be determined that the similarity is greater, and as the distanceincreases, it may be determined that the similarity is smaller.

To train such a metric learning artificial neural network, a dataset maybe artificially generated. Therefore, even when there is no change inthe current road information compared with the road information on theroad-layout map, the changed-reflected data may be artificiallygenerated, and accordingly, the artificial neural network can be trainedusing the generated data. According to an embodiment, as illustrated inFIG. 8, a negative image may be generated by changing 862, removing 864or adding 866 an object into a projected road-layout map 840corresponding to a captured road image 820. For example, when an object842 representing a left turn arrow 822 is present in a specific regionon the projected road-layout map 840 corresponding to the captured roadimage, a negative image may be generated by changing the object 842representing the left turn arrow to an object 863 representing thestraight arrow or by removing the object 842 representing the left turnarrow 865. In addition, on the projected road-layout map 840corresponding to the captured road image, an object 867 representing aspeed limit road marker (speed limit) may be inserted into a specificregion where no object is present, thereby generating a negative image.

According to another embodiment, a positive image used to train themetric learning artificial neural network may be generated by adding arandom position error to a projected road-layout map corresponding tothe captured road image. For example, a positive image may be generated,to which a random error set in the range of (−1, 1) and the range of(−5, 5) for the camera rotation angle (θ) with respect to the positioncoordinate (x, y) is added. As described above, by adding the randomerror to the projected road-layout map, the metric learning artificialneural network may be robustly trained against position errors. That is,when there is a slight difference in the location of the captured roadimage and the road-layout map and the camera rotation angle, the metriclearning artificial neural network can be trained to ignore these errorsand detect the presence or absence of actual changes in the roadinformation.

FIG. 9 is a block diagram illustrating an internal configuration of thedomain adapter 440 according to an embodiment.

The domain adapter 440 may be configured to perform the domainadaptation in order to perform a metric learning between a captured roadimage and a road-layout map of different forms of data, that is, data ofdifferent domains. In order to perform this domain adaptation, thedomain adapter 440 may include two encoders 442_1 and 442_2 and onediscriminator 444, as illustrated.

The domain adapter 440 may use the artificial neural network modelillustrated in FIG. 9 to perform domain adaptation. Such an artificialneural network model may correspond to an adversarial learningartificial neural network. The adversarial learning artificial neuralnetwork for domain adaptation may be trained in conjunction with themetric learning artificial neural network included in the system 180 fordetecting changes in road information.

Since the structure of the encoders 442_1 and 442_2 of the domainadapter 440 has been described in detail with reference to FIG. 7,redundant descriptions thereof will be omitted below. According to theartificial neural network model structure illustrated in FIG. 9, thediscriminator 444 may receive the first intermediate data and the secondintermediate data from respective encoders 442_1 and 442_2. In thisexample, the received first intermediate data and the secondintermediate data may be data in the form of four-dimensional tensor.

The discriminator 444 may be configured to repeatedly perform dataconversion four times by the structure including the convolution layer,the instance normalization layer, and the rectified linear unit/leakedrectified linear unit layer in order, and perform data conversion by theconvolution layer and the average pooling (Avgpool) layer.

The discriminator 444 of the domain adapter 440 may be trained to returncloseness of the domain of the input data to the target domain as ascalar value. According to an embodiment, the first encoder 442_1 andthe second encoder 442_2 may be trained such that an output value of thediscriminator for the first intermediate data is a first value, and anoutput value of the discriminator for the second intermediate data is asecond value, while the discriminator 444 may be trained such that theoutput value of the discriminator for first intermediate data is thesecond value, and the output value of the discriminator for secondintermediate data is the first value. For example, the first encoder442_1 may be trained such that the output value of the discriminator 444for the first intermediate data is 1, and the second encoder 442_2 maybe trained such that the output value of the discriminator for thesecond intermediate data is 0. The discriminator 444 may also be trainedsuch that an output value of the discriminator for the firstintermediate data is 0, and an output value of the discriminator for thesecond intermediate data is 1.

The discriminator 444 may calculate the domain difference between thefirst intermediate data and the second intermediate data. According toan embodiment, the difference between the expected value of the outputvalue of the discriminator 444 for the first intermediate data and theexpected value of the output value of the discriminator 444 for thesecond intermediate data may be calculated as the domain difference. Theadversarial learning artificial neural network included in the domainadapter 440 may be trained so that the difference between the domainscalculated by the discriminator 444 is minimized.

According to the difference between the domains calculated by thediscriminator 444, the margin (m) in the triplet loss of the metriclearning artificial neural network included in the similarity determinermay be adjusted. The adaptive margin (m{circumflex over ( )}′) adjustedaccording to the domain difference may be expressed by the followingformula.

m′=m×exp(−|∇α|/β)

where m is the triplet loss margin, α is the domain difference, β is theattenuated level, and m′ is the adaptive margin.

For example, since the adversarial learning artificial neural networkincluded in the domain adapter 440 is trained such that the domaindifference is small, when the domain difference is large at thebeginning of the training process (when the change in the domaindifference is large), the value of the adaptive margin (m′) is set to besmall, so that the metric learning artificial neural network can betrained even with a small loss. On the other hand, when the domaindifference is small in the latter half of the training process (when thechange in the domain difference is small), the adaptive margin (m′) mayconverge to a margin (m) of a certain value designed for training of themetric learning artificial neural network.

FIG. 10 is a block diagram illustrating an internal configuration of achange region detector 502 according to an embodiment.

The change region detector 502 may include a normalization unit 1002, anassociation unit 1004, and a change value map generator 1006. The changeregion detector 502 may generate a change value map 1008 including amask indicating a region where a change occurs between the captured roadimage and the projected road-layout map, and this function may beperformed through an artificial neural network. In this example, thechange value map 1008 may have the same resolution as the captured roadimage or the projected road-layout map.

In order to train the artificial neural network for generating thechange value map 1008 including the mask indicating a region where thechange occurs between the captured road image and the projectedroad-layout map, a ground truth mask may be defined using the projectedroad-layout map M and the artificially changed image S. For example, theground truth mask (l^(gt)) may be defined as follows

l ^(gt)=Π(M≠S)

The change region detector 502 may receive third intermediate data andfourth intermediate data from the feature extractor of the similaritycalculator. Here, the third intermediate data and the fourthintermediate data may correspond to the last intermediate feature tensoramong the intermediate feature tensors calculated in the step prior tothe global pooling step in the process of extracting the features of thecaptured road image and the projected road-layout map by the featureextractor.

The normalization unit 1002 may perform instance-wise normalization onthe input third intermediate data and fourth intermediate data. Theassociation unit 1004 may concatenate the third intermediate data andthe fourth intermediate data and input the concatenated data to thechange value map generator 1006. Here, the change value map generator1006 may include a U-net structure that outputs a result in the samesize as the input size. In one embodiment, the U-net architecture mayinclude a contracting path and an expansive path, as is known. Thecontracting path may include a typical convolutional network, andspecifically, may include a plurality of unpadded convolutions, arectified linear unit (ReLU) for downsampling, and a max poolingoperation. The extension path may include upsampling of the feature map,a plurality of upconvolutions, and ReLU.

FIG. 11 is an exemplary diagram illustrating a change value map 1140generated according to a captured road image and a projected road-layoutmap by the system 180 for detecting changes in road informationaccording to an embodiment.

In order to effectively display the changes in road information in thesystem 180 for detecting changes in road information, the captured roadimage, the projected road-layout map, and/or the change value map of theroad information may be superimposed and output as one image. Forexample, as illustrated in FIG. 11, an image 1120 may be generated, inwhich the projected road-layout map is superimposed on the captured roadimage, such that the objects included in the projected road-layout mapare superimposed on the actually captured road image.

The system 180 for detecting changes in road information may generatethe change value map 1140 indicating a region with changes in the roadinformation, based on the captured road image and the road-layout map.The change value map 1140 may include a mask generated by masking (1142,1144, 1146) the changed region of the road information. As illustrated,when the change value map 1140 is generated, the system 180 fordetecting changes in road information superimposes the change value map1140 on the captured road image and displays the result, so that thechanged region of the map information may be effectively displayed onthe captured road image.

For example, as illustrated, when an object corresponding to a left turnarrow 1122, a straight arrow 1124, and a straight/right turn arrow 1126appearing in the captured road image is not displayed on the projectedroad-layout map, a change value map may be generated by detecting theabsence of the object in the corresponding region and masking (1142,1144, 1146) the corresponding region of change.

FIG. 12 is an exemplary diagram illustrating a change value map 1240generated according to the captured road image and the projectedroad-layout map by the system 180 for detecting changes in roadinformation according to another embodiment.

As illustrated, in the image 1220 in which the captured road image andthe projected road-layout map are superimposed, when the previouslystored objects 1222, 1224, 1226 are present on the projected road-layoutmap for a region where there is no road marker displayed on the capturedroad image, the system 180 for detecting changes in road information maydetect the corresponding region as a region of change and generate thechange value map 1240 by masking (1242 and 1244) the detected region ofchange.

In FIGS. 11 and 12 described above, the change value maps 1140 and 1240only illustrate an example where the object present on the road-layoutmap is lost from the captured road image or where the object not presenton the road-layout map is added, but the present disclosure is notlimited thereto. For example, in the corresponding region of a projectedroad-layout map and a captured road image, when another object otherthan a previously present object in the projected road-layout map ispresent in the captured road image, the system 180 for detecting changesin road information may detect the corresponding region as the region ofchange, and generate the change value map by masking the detected regionof change. As another example, when an object detected in a capturedroad image corresponds to an object that is not defined on theroad-layout map, the system 180 for detecting changes in roadinformation may detect the region where the object is detected as theregion of change, and generate the change value map by masking thedetected region of change.

FIG. 13 is a graph 1300 illustrating a result of a performance ofdetecting changes in road information of the system 180 for detectingchanges in road information according to an embodiment.

The inventors of the present invention acquired 20,000 captured roadimages and projected road-layout maps in order to train the system 180for detecting changes in road information implemented according to thepresent disclosure. Specifically, in the training of the system 180 fordetecting changes in road information, images of road regions withoutchange in urban regions were used, and negative images were artificiallygenerated using the method described above. The encoder 442, thediscriminator 444, and the feature extractor 462 included in the system180 for detecting changes in road information were trained for 30 epochsusing Adam solver with an initial learning rate of 0.001 and a batchsize of 4. In addition, the change region detector 502 included in thesystem 180 for detecting changes in road information was trained for 150epochs with an initial learning rate of 0.0001, and the rest of thehyperparameters were maintained the same. The margin (m) and attenuationlevel parameter Watt) to be applied to the triplet loss function wereset to 0.4 and 0.1, respectively. A random noise in the range of ±1 mand in the range of ±5° at the camera rotation angle was added to thelocation coordinates of the captured road image or the road-layout mapused in training the system 180 for detecting changes in roadinformation.

In order to quantitatively analyze the result using the system 180 fordetecting changes in road information of the present disclosure,performance evaluation was basically conducted using the mean averageprecision score (mAP score). Specifically, mAPr can represent theperformance in terms of image retrieval of the system 180 for detectingchanges in road information, and can evaluate how well it can perform acomparison between the captured road images and the road-layout mapsbelonging to different domains from each other. Specifically, mAPr mayevaluate the minimum number of times that the system 180 for detectingchanges in road information can retrieve for the road-layout mapcorresponding to a query from the entire road layout map dataset whensetting the captured road image as a query. mAPr was calculated byaveraging the values over the entire test dataset.

mAPs can represent the performance of the system 180 for detectingchanges in road information in terms of change detection, and canevaluate how well it can find the change occurring on the road-layoutmap. Specifically, the mAPs may evaluate the minimum number of timesthat the system 180 for detecting changes in road information canretrieve the road-layout map corresponding to a query from the set ofartificially generated change maps when setting the captured road imageas a query. The mAPs was calculated by averaging values over the entiretest dataset.

In order to quantitatively evaluate or analyze the performance ofdetecting changes in road information of the system 180 for detectingchanges in road information, approximately 4,000 captured images andprojected road-layout maps for a road region without change in stillanother urban region were acquired, and an average of 40 artificiallychanged masks were generated for each image.

TABLE 1 <PERFORMANCE CHANGE FOR EACH CONFIGURATION> Configuration mAPrmAPs Baseline 0.37 0.60 +Adversarial learning 0.43 0.59 +Attenuatedmargin 0.51 0.71

Table 1 above represents the performance of the baseline (r baselinerow) including the metric learning artificial neural network modelstructure, the performance of the configuration (+ adversarial learningrow) further including the adversarial learning artificial neuralnetwork model structure in addition to the baseline, and the performanceof the configuration (+ attenuation margin row) further including theadversarial learning artificial neural network model structure inaddition to the baseline and applying the adaptive margin as the marginof the triplet loss function of the metric learning artificial neuralnetwork, as values of the mAPr and the mAPs, respectively, in the system180 for detecting changes in road information.

The mAPr value was calculated as 0.37 in the baseline, 0.43 in theconfiguration further including the adversarial learning, and 0.51 inthe configuration applying the adaptive margin. According to thisperformance value, it can be seen that the performance value increasesor improves as other configurations are added to the baseline in thesystem 180 for detecting changes in road information. Meanwhile, themAPs value was calculated as 0.60 in the baseline, 0.59 in theconfiguration further including the adversarial learning, and 0.71 inthe configuration applying the adaptive margin. Accordingly, it can beseen that the performance is improved when the adaptive margin isapplied to the baseline in the system 180 for detecting changes in roadinformation.

In view of these results, it can be seen that, in the system 180 fordetecting changes in road information, the configuration (+ adversariallearning) including the adversarial learning artificial neural networkin addition to the baseline, and the configuration (+ attenuated margin)applying the adaptive margin to the baseline has more improvedperformance than the baseline including the metric learning artificialneural network model structure. In particular, since the mAPs are mainlydesigned to reflect the change detection performance of the system 180for detecting changes in road information, while mAPr is closely relatedto the domain adaptation of the system for detecting changes in roadinformation, the advantage of the configuration of adding the adaptivelearning artificial neural network is mainly exhibited by the mAPrvalue.

TABLE 2 <mAPs SCORES BY EACH TYPE OF CHANGE IN ROAD INFORMATION> Type ofChange Category Addition Removal Class change Road Lane — 0.86 0.63Arrow marker 0.87 0.83 0.50 Other road 0.91 0.90 0.57 information

Table 2 above represents mAPs scores for the removal of lanes, change ofclass; insertion and removal of arrow marker, change of class; additionand removal of other road information and change of class, in the system180 for detecting changes in road information. According to the valuesin Table 2, the system 180 for detecting changes in road informationaccording to the present disclosure shows good overall performance inall cases, but Class change of object shows lower performance thanAddition and Removal. Accordingly, in detecting changes in roadinformation by the system 180 for detecting changes in road information,it can be seen that the class change of the object is relativelydifficult to detect compared to the removal or addition of the object.

FIG. 13 illustrates mAP scores according to noise levels with the graph1300. Specifically, FIG. 13 is a graph showing mAPr scores and mAPsscores according to noise levels, respectively, when there is noiseabout the location coordinates of the map data input to the system 180for detecting changes in road information, or noise about the camerarotation angle, or noise about both the location coordinate and thecamera rotation angle. As illustrated in the graph, even when there isnoise in the range of ±1 m in the position coordinate and/or noise inthe range of ±5° in the camera rotation angle, which is a general errorthat can occur on the input map data, the system 180 for detectingchanges in road information can maintain about 80% of the performancefor map data without noise.

FIG. 14 is a flowchart illustrating a method for detecting changes inroad information according to an embodiment.

As illustrated, a method 1400 for detecting changes in road informationmay start at S1420 of converting, by the domain adapter 440, a capturedroad image and a projected road-layout map into the first intermediatedata and the second intermediate data of the same feature space,respectively. For example, by using two encoders 442_1 and 442_2included in the domain adapter 440, the captured road image and theprojected road-layout map may be converted into intermediate data in theform of four-dimensional tensor, respectively. Here, the projectedroad-layout map may be generated by receiving the location informationand the camera parameters of the captured road image from the camera 140by the map projector 420, receiving the road-layout map corresponding tothe location information, which is received from the digital map DB 120,and projecting the received road-layout map onto the image plane of thecamera 140 based on the received location information and cameraparameters.

According to an embodiment, the captured road image may be convertedinto the first intermediate data by the first encoder 442_1, and theprojected road-layout map may be converted into the second intermediatedata by the second encoder 442_2. In addition, the domain differencebetween the first intermediate data and the second intermediate data maybe calculated by the discriminator 444. Here, the first encoder 442_1and the second encoder 442_2 may be trained to minimize the domaindifference calculated by the discriminator 444.

According to another embodiment, the first encoder 442_1 and the secondencoder 442_2 of the domain adapter 440 may be trained such that theoutput value of the discriminator 444 for the first intermediate data isthe first value, and the output value of the discriminator for thesecond intermediate data is the second value, and the discriminator maybe trained such that the output value of the discriminator for the firstintermediate data is the second value, and the output value of thediscriminator for the second intermediate data is the first value.

Thereafter, at S1440, the similarity between the captured road image andthe projected road-layout map may be calculated based on the firstintermediate data and the second intermediate data by the similaritydeterminer 460. That is, the similarity determiner 460 may quantify howsimilar the captured road image and the projected road-layout map are.According to an embodiment, by the metric learning artificial neuralnetwork, the features of the captured road image and the projectedroad-layout map may be extracted based on the first intermediate dataand the second intermediate data, and the distance between the extractedfeatures may be calculated as the similarity between the captured roadimage and the projected road-layout map. The metric learning artificialneural network may be trained to calculate the similarity between thecaptured road image and the projected road-layout map using the tripletloss function that enforces the similarity between the captured roadimage and the positive image which is the projected road-layout mapcorresponding to the captured road image to be greater by a margin thanthe similarity between the captured road image and the negative imagewhich is the projected road-layout map not corresponding to the capturedroad image.

According to an embodiment, the margin used herein may be adjustedaccording to the domain difference between the first intermediate dataand the second intermediate data calculated by the domain adapter 440.According to another embodiment, the margin may be adjusted according tothe difference between the expected value of the output value of thediscriminator 444 for the first intermediate data and the expected valueof the output value of the discriminator for the second intermediatedata, which is calculated with the domain difference by the domainadapter 440.

In addition, the positive image used to train the metric learningartificial neural network may be generated by adding the random positionerror to the projected road-layout map corresponding to the capturedroad image, and the negative image used thereto may be generated byadding, removing, or changing an object in the projected road-layout mapcorresponding to the captured road image. In the training of the metriclearning artificial neural network, an image in which anobject-displaying region in either the positive image or the negativeimage is occluded by a dynamic object in the captured road image may notbe used.

Finally, at S1460, it is possible to detect, by the change detector 480,the presence or absence of changes in the road information on theprojected road-layout map based on the calculated similarity. Accordingto an embodiment, by the similarity determiner 460, the thirdintermediate data and the fourth intermediate data may be extracted fromthe first intermediate data and the second intermediate data,respectively, and by the change region detector 502, the change valuemap having the same resolution as the captured road image or theprojected road-layout map may be extracted based on the thirdintermediate data and the fourth intermediate data. This change valuemap may be superimposed on the captured road image or the projectedroad-layout map and displayed.

FIG. 15 is a flowchart illustrating a method for training an artificialneural network for detecting changes in road information according to anembodiment.

As illustrated, the method 1500 for training an artificial neuralnetwork for detecting changes in road information may start at S1520 oftraining an adversarial learning artificial neural network to performdomain adaptation between the captured road image and the projectedroad-layout map by converting the captured road image and the projectedroad-layout map into the first intermediate data and the secondintermediate data of the same feature space, respectively.

According to an embodiment, the captured road image may be convertedinto the first intermediate data by the first encoder 442_1, theprojected road-layout map may be converted into the second intermediatedata by the second encoder 442_2, and the domain difference between thefirst intermediate data and the second intermediate data may becalculated by the discriminator 444. Here, the first encoder 442_1 andthe second encoder 442_2 may be trained to minimize the domaindifference calculated by the discriminator 444.

According to another embodiment, the first encoder 442_1 and the secondencoder 442_2 may be trained such that the output value of thediscriminator 444 for the first intermediate data is the first value,and the output value of the discriminator for the second intermediatedata is the second value, and the discriminator may be trained such thatthe output value of the discriminator for the first intermediate data isthe second value, and the output value of the discriminator for thesecond intermediate data is the first value. Then, at S1540, the metriclearning artificial neural network may be trained to extract thefeatures of the captured road image and the projected road-layout mapbased on the first intermediate data and the second intermediate data,and calculate the distance between the extracted features as thesimilarity between the captured road image and the projected road-layoutmap. According to an embodiment, the metric learning artificial neuralnetwork may be trained to calculate the similarity between the capturedroad image and the projected road-layout map using the triplet lossfunction that enforces the similarity between the captured road imageand the positive image which is the projected road-layout mapcorresponding to the captured road image to be greater by a margin thanthe similarity between the captured road image and the negative imagewhich is the projected road-layout map not corresponding to the capturedroad image. Here, the used margin may be adjusted according to thedomain difference between the calculated first intermediate data and thesecond intermediate data.

In addition, in order to train the metric learning artificial neuralnetwork, the positive image may be generated by adding a random positionerror to the projected road-layout map corresponding to the capturedroad image, and the negative image may be generated by adding, removing,or changing an object in the projected road-layout map corresponding tothe captured road image. An image in which an object-displaying regionin either the positive image or the negative image is occluded by adynamic object in the captured road image may not be used.

Finally, at S1560, based on the calculated similarity, the changedetection artificial neural network may be trained to detect thepresence or absence of changes in the road information on the projectedroad-layout map. A dataset for training the artificial neural networktrained in each step of the method 1500 for training an artificialneural network for detecting changes in road information may beartificially generated.

In the flowchart illustrated in FIG. 15, although the steps of trainingeach artificial neural network are illustrated to be performed in order,the present disclosure is not limited thereto, and each artificialneural network may be trained in a different order from that of FIG. 15or may be simultaneously trained.

The method for detecting changes in road information described above maybe implemented as a computer-readable code on a computer-readablerecording medium. The recording medium may continuously store a programexecutable by a computer or temporarily store a program for execution ordownload. In addition, the medium may be a variety of recording means orstorage means in a form in which a single piece of hardware or severalpieces of hardware are combined, but is not limited to a medium directlyconnected to any computer system, and may be present on a network in adistributed manner. Examples of media include magnetic media such ashard disks, floppy disks, and magnetic tape, optical media such asCD-ROMs and DVDs, magnetic-optical media such as floptical disks, a ROM,a RAM, a flash memory, and so on, and may be devices configured to storeprogram instructions. In addition, examples of other media also includean app store that distributes applications, a site that supplies ordistributes various software, and a recording medium or a storage mediummanaged by a server.

The methods, operations, or techniques of this disclosure may beimplemented by various means. For example, these techniques may beimplemented in hardware, firmware, software, or a combination thereof.Those skilled in the art will further appreciate that the variousillustrative logical blocks, modules, circuits, and algorithm stepsdescribed in connection with the disclosure herein may be implemented inelectronic hardware, computer software, or combinations of both. Toclearly illustrate this interchangeability of hardware and software,various illustrative components, blocks, modules, circuits, and stepshave been described above generally in terms of their functionality.Whether such a function is implemented as hardware or software variesdepending on design requirements imposed on the particular applicationand the overall system. Those skilled in the art may implement thedescribed functions in varying ways for each particular application, butsuch implementation should not be interpreted as causing a departurefrom the scope of the present disclosure.

In a hardware implementation, processing units used to perform thetechniques may be implemented in one or more ASICs, DSPs, digital signalprocessing devices (DSPDs), programmable logic devices (PLDs), fieldprogrammable gate arrays (FPGAs), processors, controllers,microcontrollers, microprocessors, electronic devices, other electronicunits designed to perform the functions described in the disclosure,computer, or a combination thereof.

Accordingly, various example logic blocks, modules, and circuitsdescribed in connection with the disclosure may be implemented orperformed with general purpose processors, DSPs, ASICs, FPGAs or otherprogrammable logic devices, discrete gate or transistor logic, discretehardware components, or any combination of those designed to perform thefunctions described herein. The general purpose processor may be amicroprocessor, but in the alternative, the processor may be anyconventional processor, controller, microcontroller, or state machine.The processor may also be implemented as a combination of computingdevices, for example, a DSP and microprocessor, a plurality ofmicroprocessors, one or more microprocessors associated with a DSP core,or any other combination of the configurations.

In the implementation using firmware and/or software, the techniques maybe implemented with instructions stored on a computer readable medium,such as random access memory (RAM), read-only memory (ROM), non-volatilerandom access memory (NVRAM), programmable read-only memory (PROM),erasable programmable read-only memory (EPROM), electrically erasablePROM (EEPROM), flash memory, compact disc (CD), magnetic or optical datastorage devices, and the like. The instructions may be executable by oneor more processors, and may cause the processor(s) to perform certainaspects of the functions described in the present disclosure.

When implemented in software, the techniques may be stored on a computerreadable medium as one or more instructions or codes, or may betransmitted through a computer readable medium. The computer readablemedia include both the computer storage media and the communicationmedia including any medium that facilitates the transfer of a computerprogram from one place to another. The storage media may also be anyavailable media that may be accessed by a computer. By way ofnon-limiting example, such a computer readable medium may include RAM,ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storageor other magnetic storage devices, or any other media that can be usedto transfer or store desired program code in the form of instructions ordata structures and can be accessed by a computer. Also, any connectionis properly referred to as a computer readable medium.

For example, when the software is transmitted from a website, server, orother remote sources using coaxial cable, fiber optic cable, twistedpair, digital subscriber line (DSL), or wireless technologies such asinfrared, wireless, and microwave, the coaxial cable, the fiber opticcable, the twisted pair, the digital subscriber line, or the wirelesstechnologies such as infrared, wireless, and microwave are includedwithin the definition of the medium. The disks and the discs used hereininclude CDs, laser disks, optical disks, digital versatile discs (DVDs),floppy disks, and Blu-ray disks, where disks usually magneticallyreproduce data, while discs optically reproduce data using a laser. Thecombinations described above should also be included within the scope ofthe computer readable media.

The software module may reside in RAM memory, flash memory, ROM memory,EPROM memory, EEPROM memory, registers, hard disk, removable disk,CD-ROM, or any other form of storage medium known. An exemplary storagemedium may be connected to the processor, such that the processor mayread or write information from or to the storage medium. Alternatively,the storage medium may be integrated into the processor. The processorand the storage medium may exist in the ASIC. The ASIC may exist in theuser terminal. Alternatively, the processor and storage medium may existas separate components in the user terminal.

Although the embodiments described above have been described asutilizing aspects of the currently disclosed subject matter in one ormore standalone computer systems, the present disclosure is not limitedthereto, and may be implemented in conjunction with any computingenvironment, such as a network or distributed computing environment.Furthermore, aspects of the subject matter in this disclosure may beimplemented in multiple processing chips or devices, and storage may besimilarly influenced across a plurality of devices. Such devices mayinclude PCs, network servers, and portable devices.

Although the present disclosure has been described in connection withsome embodiments herein, various modifications and changes can be madewithout departing from the scope of the present disclosure, which can beunderstood by those skilled in the art to which the present disclosurepertains. Further, such modifications and changes are intended to fallwithin the scope of the claims appended herein.

What is claimed is:
 1. A method for detecting changes in roadinformation performed by at least one processor of a computer system,the method comprising: converting, by a domain adapter, a captured roadimage and a projected road-layout map into first intermediate data andsecond intermediate data of a same feature space, respectively;calculating, by a similarity determiner, a similarity between thecaptured road image and the projected road-layout map based on the firstintermediate data and the second intermediate data; and detecting, by achange detector, presence or absence of changes in road information onthe projected road-layout map based on the calculated similarity.
 2. Themethod according to claim 1, wherein, prior to the converting thecaptured road image and the projected road-layout map into the firstintermediate data and the second intermediate data of the same featurespace, respectively, receiving location information and cameraparameters of the captured road image from a camera; receiving aroad-layout map corresponding to the received location information froma digital map database; and generating, by a map projector, theprojected road-layout map by projecting the received road-layout maponto an image plane of the camera based on the received locationinformation and the received camera parameters.
 3. The method accordingto claim 1, wherein the calculating of the similarity between thecaptured road image and the projected road-layout map includesextracting, by a metric learning artificial neural network, features ofthe captured road image and the projected road-layout map based on thefirst intermediate data and the second intermediate data, andcalculating a distance between the extracted features as the similarity.4. The method according to claim 3, wherein the metric learningartificial neural network is trained to calculate the similarity betweenthe captured road image and the projected road-layout map using atriplet loss function that enforces a similarity between the capturedroad image and a positive image which is the projected road-layout mapcorresponding to the captured road image to be larger by a margin than asimilarity between the captured road image and a negative image which isthe projected road-layout map not corresponding to the captured roadimage.
 5. The method according to claim 4, wherein the margin isadjusted according to a domain difference between the first intermediatedata and the second intermediate data calculated by the domain adapter.6. The method according to claim 4, wherein the positive image isgenerated by adding a random position error to the projected road-layoutmap corresponding to the captured road image, and the negative image isgenerated by adding, removing, or changing an object in the projectedroad-layout map corresponding to the captured road image.
 7. The methodaccording to claim 6, wherein, in the training of the metric learningartificial neural network, an image in which an object-displaying regionin either the positive image or the negative image is occluded by adynamic object in the captured road image is not used.
 8. The methodaccording to claim 1, wherein the converting of the captured road imageand the projected road-layout map into the first intermediate data andthe second intermediate data of the same feature space, respectively,includes: converting, by a first encoder, the captured road image intothe first intermediate data; converting, by a second encoder, theprojected road-layout map into the second intermediate data; andcalculating, by a discriminator, a domain difference between the firstintermediate data and the second intermediate data, and the firstencoder and the second encoder are trained to minimize the domaindifference calculated by the discriminator.
 9. The method according toclaim 8, wherein the first and second encoders are trained such that anoutput value of the discriminator for the first intermediate data is afirst value, and an output value of the discriminator for the secondintermediate data is a second value, and the discriminator is trainedsuch that the output value of the discriminator for the firstintermediate data is the second value, and the output value of thediscriminator for the second intermediate data is the first value. 10.The method according to claim 8, wherein the similarity determinerincludes a metric learning artificial neural network which is trained tocalculate the similarity between the captured road image and theprojected road-layout map using a triplet loss function that enforces asimilarity between the captured road image and a positive image which isa projected road-layout map corresponding to the captured road image tobe greater by a margin than a similarity between the captured road imageand a negative image which is a projected road-layout map notcorresponding to the captured road image, the calculating of thesimilarity between the captured road image and the projected road-layoutmap includes extracting, by a metric learning artificial neural network,features of the captured road image and the projected road-layout mapbased on the first intermediate data and the second intermediate data,and calculating a distance between the extracted features as thesimilarity, the calculating of the domain difference between the firstintermediate data and the second intermediate data includes calculating,by the domain adapter, a difference between an expected value of anoutput value of the discriminator for the first intermediate data and anexpected value of an output value of the discriminator for the secondintermediate data as the domain difference, and the margin is adjustedaccording to the domain difference between the first intermediate dataand the second intermediate data calculated by the domain adapter. 11.The method according to claim 1, wherein the calculating of thesimilarity between the captured road image and the projected road-layoutmap includes extracting, by the similarity determiner, thirdintermediate data and fourth intermediate data, respectively, from thefirst intermediate data and the second intermediate data, and thedetecting of the presence or absence of changes in the road informationon the projected road-layout map based on the calculated similarityincludes generating, by a change region detector, a change value maphaving a same resolution as the captured road image or the projectedroad-layout map based on the third intermediate data and the fourthintermediate data.
 12. A method for training an artificial neuralnetwork for detecting changes in road information, the method performedby at least one processor of a computer system, the method comprising:training an adversarial learning artificial neural network to performdomain adaptation between a captured road image and a projectedroad-layout map by converting the captured road image and the projectedroad-layout map into first intermediate data and second intermediatedata of a same feature space, respectively; training a metric learningartificial neural network to extract features of the captured road imageand the projected road-layout map based on the first intermediate dataand the second intermediate data, and calculate a distance between theextracted features as a similarity between the captured road image andthe projected road-layout map; and training a change detectionartificial neural network to detect presence or absence of changes inroad information on the projected road-layout map based on thecalculated similarity.
 13. The method according to claim 12, wherein thetraining of the metric learning artificial neural network includestraining the metric learning artificial neural network so that themetric learning artificial network is configured to calculate asimilarity between the captured road image and the projected road-layoutmap using a triplet loss function that enforces a similarity between thecaptured road image and a positive image which is a projectedroad-layout map corresponding to the captured road image to be greaterby a margin than a similarity between the captured road image and anegative image which is a projected road-layout map not corresponding tothe captured road image.
 14. The method according to claim 13, whereinthe training of the metric learning artificial neural network includes:calculating a domain difference between the first intermediate data andthe second intermediate data; and adjusting the margin according to thedomain difference.
 15. The method according to claim 13, wherein thetraining of the metric learning artificial neural network includes:generating the positive image by adding a random position error to aprojected road-layout map corresponding to the captured road image; andgenerating the negative image by adding, removing, or changing an objectin the projected road-layout map corresponding to the captured roadimage.
 16. The method according to claim 15, wherein the training of themetric learning artificial neural network includes deleting an image inwhich an object-displaying region either in the positive image or thenegative image is occluded by a dynamic object in the captured roadimage.
 17. The method according to claim 12, wherein the training of theadversarial learning artificial neural network includes: converting, bya first encoder, the captured road image into the first intermediatedata; converting, by a second encoder, the projected road-layout mapinto the second intermediate data; calculating, by a discriminator, adomain difference between the first intermediate data and the secondintermediate data; and training the first encoder and the second encoderto minimize the domain difference calculated by the discriminator. 18.The method according to claim 17, wherein the training of theadversarial learning artificial neural network includes: training thefirst encoder and the second encoder such that an output value of thediscriminator for the first intermediate data is a first value, and anoutput value of the discriminator for the second intermediate data is asecond value; and training the discriminator such that the output valueof the discriminator for the first intermediate data is the secondvalue, and the output value of the discriminator for the secondintermediate data is the first value.
 19. A system for detecting changesin road information, comprising: a domain adapter configured to converta captured road image and a projected road-layout map into firstintermediate data and second intermediate data of a same feature space,respectively, to perform domain adaptation between the captured roadimage and the projected road-layout map; a similarity determinerconfigured to calculate a similarity between the captured road image andthe projected road-layout map based on the first intermediate data andthe second intermediate data; and a change detector configured to detectpresence or absence of changes in road information on the projectedroad-layout map based on the calculated similarity.
 20. The systemaccording to claim 19, further comprising: a map projector configured toreceive location information and camera parameters of the captured roadimage from a camera, receive a road-layout map corresponding to thereceived location information from a digital map database, and projectthe received road-layout map onto an image plane of the camera based onthe received location information and the received camera parameters togenerate the projected road-layout map.